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The focus of this current grant was to continue our work which focused on the 
manner in which psychophysiological markers can be used to index hazardous states 
of awareness and to explore the feasibility of developing on-line systems that utilize 
real time feedback to modify on-going behavioral processes. In this work we have 
incorporated a multifaceted approach which includes psychophysiological, subjective, 
and performance based measures. We have considered this from both an internal and 
external perspective as reflected in work from a variety of labs. The external 
perspective has emphasized the task demands or workload as reflected by time 
pressure or complexity of tasks. However, previous research has shown that task 
demand and complexity cannot be modeled in a simple linear fashion. Thus, 
examinations of internal factors which can be reflected in a variety of 
psychophysiological measures offers an alternative window in terms of cognitive 
processing. In addition to the task requirements, individuals are also sensitive to the 
needs of the organisms some of which are preprogrammed through our evolutionary 
history. In this sense, all individuals engage constantly in a variety of multitasking 
situations which must be taken into account in developing models that reflect 
circumstances in which individuals work at both optimal and suboptimal levels. Lapses 
of awareness has been an important theme in our work. In addition to creating 


situations that vary in terms of task demands, we have also varied individual 
differences factors such as anxiety, dissociative tendencies, or hypnotic suggestibility 
to model situations of stress, spacing out, or increased suggestibility and narrowed 
focus. Since there exists a close connection between anxiety and stress responses as 
well as between anxiety and internal distracting thoughts, we emphasized the individual 
difference of anxiety in our current work. Further, given the lack of one to one mapping 
between most physiological measures and complex cognitive, emotional, and motor 
responses, we have conceptually sought to articulate cognitive primitives which reflect 
basic environmental-organism responses. These basic cognitive building blocks can 
also help us better understand the manner in which the organism internally is 
modulated by its own responses in complex situations. Using this understanding, we 
have considered the feasibility of using these internals processes to create an 
architecture of situational control. In this progress report, we describe work from three 
studies designed to develop and validate psychophysiologcial methods related to the 
detection of hazardous states of awareness and report on the development of a 
biocybernetic system. 

EEG Theta Activity and Error Detection 

Frontal midline theta was initially noted during the performance of mental tasks 
such as mental arithmetic and that it increases with task difficulty. Theta has also been 
linked to an event related potential, that of error-related negativity or ERN that peaks 
approximately 100 ms after an individual makes an error. This activity appears related 
to error monitoring in relation to action regulation. In terms of its source, a variety of 


studies suggest that the anterior cingulate is the source of this 4-7 Hz midline activity. 
Broca in the 1870s first described the “le grand lobe limbique” or limbic system with the 
dorsal part being the cingulated cortex. Papez in the 1930s saw this structure as the 
receptive organ for the experience of emotion. The anterior cingulate receives 
information from the amygdala which is assumed to be fear related as well as show 
neuronal activity in relation to performance reward and error. Currently, the anterior 
cingulated is seen to monitor performance, including errors and reward, and to adjust 
behavior to optimize payoff. The purpose of the 4-7 Hz theta activity appears to be that 
of fostering entrainment in an interregional manner that allows for limbic circuits to 
exert regulatory control on cortical networks and thus facilitate the reestablishment of 

executive control following an error condition. 

In order to examine the feasibility of using theta activity to index performance, 
we used a video game to study electrophysiological markers associated with successful 
and failed performance. In this study individuals performed a visuo-motor task 
contained within the computer game, “Frustrated Maze”. The basic task was to drag a 
ball using the mouse along a narrow mazelike path from the starting point to the goal 
without touching the “maze wall” and other obstacles. We had subjects play the game 
both under time pressure and non-time pressure conditions. During the games we 
recorded EEG activity from 15 electrode sites (Fpl, Fp2, Fz, F3, F4, Cz, C3, C4, Pz, 

P3, P4, T7, T8, 01, 02) referenced to linked mastoids. Comparing the successful with 
the unsuccessful trials, we found three major results. First, there was a widely 
distributed reduction of fast alpha (peak of approximately 10.5 Hz) over frontal-central 
and parietal areas and centrally localized reduction of “mu” over the somatomotor 


cortex during successful trials. Second, there was localized enhancement of frontal 
midline theta (Fm theta) and midline gamma in conjunction with successful trials. And 
third, these findings were more pronounced in the time pressure condition. We assume 
that the reduction in alpha, especially during the time pressure conditions, reflects an 
increase in task demands and complexity as has been noted in other studies. Theta, on 
the other hand, showed an increase in power during successful trials. Overall, this 
research lends support to the idea of an index of engagement suggest by previous 
NASA research and suggests an additional formulation of this index using frontal 
midline theta. 

Tremor 

Tremor is defined as small involuntary fluctuations in limb position that 
accompany voluntary postural tasks, with and without load. Traditionally, tremor has 
been divided into physiological, essential, and Pathological types. Physiological tremor 
is a “normal” tremor that appears to be caused by a complex interaction between 
physical/mechanical and neural systems. In that evidence suggests that tremor may 
have a role in the timing of the initiation of voluntary movement, it is important for our 
purposes both as potential index of temporal processing and as an index of situational 
processes such as complexity and workload. Compared to other physiological signals, 
tremor can be easily measured using an accelerometer attached to the finger. 
Mathematically, physiological tremor is linear and stochastic. Interestingly, 
physiological tremor has been shown to have two dominant frequencies in the 8-15 Hz 
range and another around 26Hz. Since the predominant frequencies of tremor appear 


similar to those of the EEG, we first considered the relationship between resting tremor 
and EEG. Next, we considered the possibility that tremor measurements might give us 
a simple window that would reflect task demands. Finally, we examined tremor is a 
population of individuals who varied in terms of anxiety. 

The procedure for all of our studies used an accelerometer taped to the left 
index finger which was connected to a Coulbourn Instruments Transducer Coupler 
Type A (Strain Gage Bridge) S72-25. EEG was recorded with a ECI Electro-cap 
System and a 21 -channel Nihon-Kohden electroencephalograph (Model 4321 B). EEG 
was recorded from the left and right frontal, central, temporal, parietal, and occipital 
sites (F 3 , F 4 , Fz, C 3 , C 4 , C z , P 3 , P 4 , Pz T 3i T 4 , T s , T 6i Ch, and 0 2 ) according to the 
international 10-20 system for electrode placement (Jasper, 1958) using a linked ear 
reference. Both horizontal and vertical eye movement was recorded. These 
physiological signals were then digitized and stored on a computer for later analysis. 
Participants were seated in front of a computer screen in a comfortable chair. 

In the first study, we examined 9 health individuals during a baseline, mental 
math task, and a relaxation task which last for 2 minutes each. Tremor was measured 
from both the left and the right hand. We performed Fast Fourier analysis (FFT) on 
these tremor time series signals. We found the tremor frequency in the left and right 
hands to be similar with a peak at approximately 10.5 Hz and 15.5 Hz in each hand. 
During the three tasks the frequency of tremor remained constant with the amplitude 
increasing between the mental math and relax task. We next examined the coherence 
between the tremor time series of the left and right hands. There was a .2 coherence at 
15.5 Hz between the two hands at baseline which decreased to .1 1 during relaxation 


and .02 during mental math. As in previous research EEG hemispheric differences 
varied according to task. The important finding of this work is that tremor is reflective of 
the task requirement and appears to be influenced by different processes during 
different tasks. 

Given that one characteristic that leads to accidents and inattention is the 
individual’s internal state, especially in term of negative affect and stress, we replicated 
our tremor study using individuals who reported high level of anxiety in their everyday 
life. The basic rationale supported by a variety of studies is that one common 
characteristic of increased stress, anxiety, or workload is that of a reduced cognitive 
capacity. In this study we examined 6 anxious individuals and 5 individuals of similar 
age and background who reported normal levels of anxiety. As previously, a FFT was 
applied to both the EEG and tremor data to determine predominant frequencies. In both 
the anxious group and the non-anxious group, the tremor data showed frequency peaks 
at approximately 10Hz and 29 Hz. Looking at the data on an individual level showed a 
great consistency among individuals suggesting that the frequency of tremor is a basic 
physiological property of human physiology. We next examined the coherence between 
the tremor and the EEG. In examining the data on an individual subject level, it became 
apparent that non-anxious individuals show a much larger coherence between finger 
tremor and EEG at 1 1 Hz. than anxious individuals. Thus, on both an individual level as 
found in this study and a situational level as found in the previous study, there appears 
to be a reduction in coherence either between tremor in each hands or between tremor 
and EEG when there is a reduction in cognitive capacity. We are continuing to validate 
tremor as a means of indexing changes in task demands. 


Startle 


The startle reflex is a set of skeletomuscular contractions viewed as a behavioral 
interrupt which prepares the organism for action. It is elicited by an intense stimulus of 
sudden onset and generally measured by its eye blink component in humans. Animal 
research suggests that the neuronal pathway of the acoustic startle pathway is primitive 
and consists of only three synapses which connect the cochlear root neurons, neurons 
in the nucleus reticularis pontis caudalis, and motorneurons in the facial motor nucleus. 

The startle response is amplified during negative experiences and reduced 
during positive ones. In the present study we build on this by presenting startle stimuli 
while individuals performed tasks that were associated with stress and effort 
(sympathetic activation) as well as tasks associated with relaxation and no cognitive 
effort (parasympathetic tasks). Since additional mechanisms such as attentional 
engagement have also been implicated in contributing to startle amplitude, we 
additionally sought to probe the attentional component by utilizing tasks that directly 
manipulate attentional focus. Thus, we had subjects perform sympathetic and 
parasympathetic tasks that varied in attention focus. 

In our study startle was measured with two 4-mm Ag/AgCI electrodes filled with 
high conductivity electrode gel and placed over the orbicularis oculi muscle below the 
left pupil. The startle stimulus consisted of a 95db fast rise time 50 millisecond white 
noise signal presented biaurally through headphones. All startle probes were 
presented at random intervals within each task, with the constraint that (a) at least 30 
seconds elapsed during any task before the presentation of the first probe and (b) a 


minimum of 30 seconds always elapsed between startle probes. For data analysis 
digitized raw EMG signals from the orbicularis oculi were amplified at a constant level 
for all participants and sampled at a lOOOhz rate 200 ms prior to and 800 ms following 
the startle stimulus. A high pass filter was used to remove slow wave activity. The EMG 
signal was half-wave rectified and integrated using Matlab software developed by our 
lab. 

We had 9 anxious and 9 non-anxious individuals perform two sympathetic tasks, 
a mental math task, and the stroop color interference task, and two parasympathetic 
tasks, watching a relaxing video of birds flying over the ocean and imagining a pleasant 
meal. Of the four tasks, two required external focus and two required internal focus. To 
determine the effects of attentional focus and autonomic nervous system activation, a 2 
(internal vs external focus-of-attention tasks) by 2 (sympathetic vs parasympathetic 
tasks) ANOVA was performed between the anxious and non-anxious groups. A main 
effect for group was found in which anxious individuals showed significantly larger 
startle amplitudes (.28 vs .17). There was also a significant main effect for type of 
attention tasks, but not for type of autonomic nervous system tasks activation. Larger 
startle responses were seen during the internal as opposed to external focus of 
attention tasks (.29 vs .16). We also asked subjects to rate their mental state during the 
initial baseline and found that the more negative they experienced their inner state, the 
larger the startles were during this period (r = .59). At this point, the startle response 
appears to be an appropriate measure to model internal negative affectivity but not the 
nature of the task in terms of stressfulness. Thus, it offers a unique method for 
differentiating task characteristics from individual state characteristics. Since the startle 


is able to model internal states, we are currently following up this study and asking 
whether predictability or control in terms of stimulus presentation reduces internal 
negative states. Although we not the completed the final analysis, it appears as if 
individuals show less negative affect in situations in which they know what will happen 
(predictability) as opposed to control (they are in change). 

Developing a Biocybernetic System 

In order to develop a biocybernetic system that would utilize psychophsyiological 
such as EEG, tremor and startle, we continued to review the existing literature on 
adaptive interfaces. The ultimate goal of this work is to improve performance in 
situations in which awareness is limited (e.g., “spacing out”, drowsiness or other 
hazardous states of awareness), pathological (e.g., attention deficit disorder) or as an 
alternative for normal manual means of control. In terms of physiological approaches to 
human-computer interfaces, there have been two major approaches. First, there have 
been attempts to train individual operators to control various aspects of their physiology 
(e.g., eye movement, EEG) which in turn can be utilized for controlling various display 
characteristics. And, second, using various forms of signal processing, there have been 
attempts to identify physiological signals associated with various cognitive states (e.g., 
attention, drowsiness) and in turn to modify displays in terms of these inferred states. 
Three academic based labs that have been historically involved in this interface work in 
relation to EEG are Donchin and his colleagues at the University of Illinois, Birbaumer 
and his colleagues at the University of Tubingen, and Wolpaw and his colleagues at 
the New York State Department of Health (Wadsworth Center). At NASA Langley, this 


work has been conducted by Alan Pope, Lance Prinzel and their colleagues. This work 
has focused on EEG index of engagement and has produced a varied of pioneering 
studies based on a closed loop system. 

We sought to extend the previous work by examining more discrete markers of 
correct and incorrect responses. We initially examined markers associated with 
successful memory perform using traditional EEG Fourier markers. In the present grant 
we have extended this using more exact signatures with more temporally precise 
measurement techniques including wavelet analysis. In this situation, the subjects were 
not temporally aware that they would make a mistake. At present both EEG gamma and 
theta band activity appear to play an important role. We have also explored the EEG 
signature when subjects know that they are supplying erroneous information as might 
be case in situations of extreme stress. Further, we conducted a series of studies which 
tend to suggest that using EEG coherence to reflect phase angle and thus direction of 
flow, there are differentially different directional vectors in terms of brain activity that 
corresponds with internal state. We are current examining these data to determine if 
task requirements are likewise reflected in EEG coherence data. 

In terms of a biocybernetic system, we developed an initial system using digital 
processing systems developed by Data Translation after long discussions with their 
engineers. We had difficulty programming this system in a manner that could analyze 
the EEG stream online and allow it to control stimulus presentations based on this 
information. It appeared that some of the features that we required had not been fully 
tested by the company when it was released to us. We have now moved to a MATLAB 
based system with utilizes National Instrument digital control cards in a real time 


environment. This system allows for ease of programming and more compatible links 
with our existing acquisition and presentation systems. This flexibility has allowed us to 
modify analytic characteristics that are applied to the EEG and to be view in real-time 
the original and resultant signal. The current design is based on simple decision rules 
but we are exploring the use of fuzzy logic and other such techniques for signal 
identification. The future goal will include decision making procedures which can 
incorporate real time information from a variety of physiological signals and external 
characteristics of the tasks. 

Summary 

Grant NAG 1 2273 continued to build on our previous work 
combining operator states and external task. In this progress report, we 
described work from three studies designed to develop and validate 
psychophysiologcial methods related to the detection of hazardous states of awareness 
and report on the development of a biocybernetic system. In specific, we describe three 
measures: EEG frontal theta; resting tremor; and the startle reflex which offer indexes 
of performance, task demands, and internal emotional state. 


